Supplementary Material for Online Adaptor Grammars with Hybrid Inference
نویسندگان
چکیده
In this document, we outline the definition of an adaptor grammar and the generative process of our model (Section 1). This allows us to define a joint distribution over adapted grammars and observations (Section 2). Uncovering the latent variables of the model (grammars and productions) requires posterior inference. We use online hybrid variational inference, which requires three components: positing a variational distribution (Section 3), deriving the mean-field updates (Section 4), and then adapting those updates into the online setting (Section 5). Section 6 serves as a reference to review all of the notation used in this document.
منابع مشابه
Hybrid Online Inference with Adaptor Grammars
Adaptor grammars are a flexible, powerful formalism for defining nonparametric, unsupervised models of grammar productions. This flexibility comes at the cost of expensive inference. We address the difficulty of inference through an online algorithm which uses a hybrid of Markov chain Monte Carlo and variational inference. We show that this inference strategy is more scalable than past approaches.
متن کاملOnline Adaptor Grammars with Hybrid Inference
Adaptor grammars are a flexible, powerful formalism for defining nonparametric, unsupervised models of grammar productions. This flexibility comes at the cost of expensive inference. We address the difficulty of inference through an online algorithm which uses a hybrid of Markov chain Monte Carlo and variational inference. We show that this inference strategy improves scalability without sacrif...
متن کاملVariational Inference for Adaptor Grammars
Adaptor grammars extend probabilistic context-free grammars to define prior distributions over trees with “rich get richer” dynamics. Inference for adaptor grammars seeks to find parse trees for raw text. This paper describes a variational inference algorithm for adaptor grammars, providing an alternative to Markov chain Monte Carlo methods. To derive this method, we develop a stick-breaking re...
متن کاملImproving nonparameteric Bayesian inference: experiments on unsupervised word segmentation with adaptor grammars
One of the reasons nonparametric Bayesian inference is attracting attention in computational linguistics is because it provides a principled way of learning the units of generalization together with their probabilities. Adaptor grammars are a framework for defining a variety of hierarchical nonparametric Bayesian models. This paper investigates some of the choices that arise in formulating adap...
متن کاملAdaptor Grammars: A Framework for Specifying Compositional Nonparametric Bayesian Models
This paper introduces adaptor grammars, a class of probabilistic models of language that generalize probabilistic context-free grammars (PCFGs). Adaptor grammars augment the probabilistic rules of PCFGs with “adaptors” that can induce dependencies among successive uses. With a particular choice of adaptor, based on the Pitman-Yor process, nonparametric Bayesian models of language using Dirichle...
متن کامل